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,  INTRODUCTION 

The  term  "sensitivity  test"  is  used  in  statistical  literature  to  denote  any 
experiment  from  which  quant al  data  (yes-no,  or  success- failure  response)  axe 
observed  as  the  intensity  of  a  stimulus  is  varied.  The  stimulus  may  be  the 
striking  velocity  of  a  projectile  and  the  response  the  penetration  or 
non-penetration  of  armor;  or  the  stimulus  may  be  the  distance  from  muzzle  to 
target  and  the  response  the  functioning  or  non-functioning  of  a  fuze,  and  so 
on.  It  Is  customary  in  the  analysis  of  such  experiments  to  use  the  integrated 
normal  distribution  as  the  model  for  describing  the  probability  of  "success" 
as  the  Intensity  of  the  stimulus  varies.  (Other  models  such  as  the  Logistics 
Curve  are  also  used  though  less  widely  than  the  Normal  Curve.) 

Analysis  of  sensitivity  data  following  the  normal  model  requires 
considerable  computation  except  in  very  special  situations.  To  facilitate 
Making  such  analyses  the  computation  of  the  maximum  likelihood  estimates  of 
the  unknown  parameters  and  related  statistics  was  programed  for  the  Bendix 
C-15-D  Computer.  The  program  is  Intended  to  be  of  assistance  to  engineering 
personnel  of  DfcPS  in  their  analyses  of  tests  of  armor  plate,  penetration  tests 
of  ammunition,  and  in  tests  of  fuzes  for  arming  distance.  The  program  would 
normally  not  be  used  to  calculate  the  ballistic  limit  for  a  small  test,  but 
could  be  used  in  the  analysis  of  a  series  of  small  tests. 

This  report  explains  the  use  of  the  program,  the  input  data,  formulas 
used,  and  output  data,  and  gives  some  comments  on  the  applicability  of  this 
type  of  analysis  to  tests  involving  a  small  number  of  rounds.  This  work 
was  carried  out  as  part  of  project  "Procedures  and  Instrumentation  on  Techniques 
for  Industrial  Testing". 


THEORY 


The  theory  of  estimation  by  the  method  of  maximum  likelihood  is  developed 
In  statistical  texts  such  as  Cramir,  Mathematical  Methods  of  Statistics,  and 
its  application  to  sensitivity  analysis  is  descriWd  in  a  number  of  publications, 
e.g.  Dixon  and  Mood  "A  Method  for  Obtaining  and  Analyzing  Sensitivity  Data" 
Journal  of  the  American  Statistical  Association,  Vol.  43  U>j$^ndrJRL 
Technical  Note  151.  A  brief  statement  of  some  results  of  the  u&ory.,'  however, 
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must  be  given  here  so  that  data  computed  by  the  machine  program  can  be  properly 
Interpreted. 


Model 


The  probability  (p)  of  success  with  stimulus  x  is  assumed  to  be 


Z~J±  a. 

( 

i  iflr 


d-i* 


where  /j_  and  (J*  are  unknown  parameters 


Let  Yj.  be  a  random  variable  taking  values  1  or  0  depending  on  whether  the 
outcome  of  the  trial  is  a  guccoss  or  failure  at  level  Xj  of  the  stimulus. 

Let  Prob  (Yj-d.)  ■  pj.  Then  Prob  (Y^-0)  »  1  -  p^. 

Estimation  of  Parameters  fjj  and  <r 

The  likelihood  equation  is 

Tl 

L  =  "J|" Pi  yi(l  -  P1)1*yi  where  pA  is  a  function  of  juf  <r. 


Solution  of  equations 


3  log  L 


d  Log  L 

3<r 


0 


A 

for /o  and  cr gives  the  maximum  likelihood  estimates  ^  and  <T . 

Since  the  two  equations  above  are  not  directly  soluble,  an  iteration  scheme 
is  employed.  The  equations  for  iteration,  in  matrix  notation,  are: 


iiuU 

3«- 
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where  f4.g,  o;  is  a  trial  value.  Solution  of  these  equations  (linear)  for  £p_ 
and act  gives  the  first  provisional  solution  as 

V4-*  -  1- 

-  0*o  +  a,ct 

The  iteration  equations  are  re-evaluated  with  f*.,  c,  replacing  /u.,  (ffl  and  a 
second  provisional  solution  is  found.  ‘  ‘ 

p.,  t 

crx  n  cr,  -► 

The  iteration  is  continued  until  A^-and  A<T  are  "small".  The  final  provisional 

solution  vlll  be  called  the  maximum  likelihood  solutions  and  denoted  ui  & 

/  * 

Estimation  of  Precision 

Maxi  was  likelihood  theory  shove  that  p  and  O'  (far  large  samples)  are 
Jointly  normally  distributed.  The  theory  also  shovs  that  any  provisional 
solution  has  the  same  asymptotic  distribution.  Since  the  distributions  are 
the  same  no  loss  in  accuracy  Is  incurred  toy  treating  the  provisional  solution 
as  the  maximum  likelihood  estimate.  In  large  samples  {jX-p  )  and  (cr-tf)  are 
approximately  normally  distributed  with  zero  means  and  covariance  matrix,  the 
Inverse  of  A  where 


PROGRAM 

This  problem  has  been  handled  as  three,  separate  programs  corresponding 
to  three  different  configurations  of  data. 

I.  Grouped  Data.  This  class  is  characterised  by  more  than  one  trial  at 
each  xi,  or  by  a  large  number  of  trials  for  which  the  xj  can  be  grouped  for 
convenience  according  to  conanon  statistical  practice. 

II.  Uhgrouped  Data.  Here  the  number  of  trials  is  nail  and  only  one 
trial  Is  conducted  at  each  x*.  In  practice  more  than  one  trial  may  have  been 
performed  at  some  values  of  x,  but  the  trials  ore  treated  as  though  the  x  values 
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are  distinct.  This  part  is  further  subdivided  because  of  a  peculiarity  of  the 
method  of  estimation  when  no  zone  or  mixed  results  ('£&)  occur.  Call  Xg  the 
largest  x*  for  which  a  failure  occurred,  and  xl  the  smallest  xj,  for  which  a 
success  occurred.  A  zone  of  mixed  results  exists  if  and  only  if  xg  >  xj,.  The 
program  for  this  situation  is  called  "Ungrouped  Data,  ZMR." 

When  xh  <  xl  there  is  no  zone  of  mixed  results  and  the  estimation  procedure 
used  in  the  foregoing  fails  because  the  estimate  of  the  scale  parameter  (<r) 
is  zero.  This  situation,  though  regarded  as  an  exception,  occurs  quite  often 
in  small  samples.  The  data  provlos  no  estimate  of  CT  j  jx.  can,  however,  be 
estimated  If  <T  is  known  or  can  be  assigned  a  reasonable  value,  say  no  less 
than  one-half  nor  more  than  twice  the  true  value.  This  program  is  labelled 
"Ungrouped  Data,  No.  ZMR." 

The  details  of  data  input,  formulas  for  machine  use,  and  of  typed  out 
results  are  contained  In  the  paragraphs  that  follow.  Although  each  program  is 
treated  separately  formulas  for  "Ungrouped  Data,  No.  ZMR"  are  given  as  modifi¬ 
cations  of  those  in  the  program  for  "Ungrouped  Data,  ZMR". 

In  tne  formulas  of  all  three  programs,  U3e  is  made  of  the  following  symbols 
not  previously  defined. 

tj,  =  (*i"M.)/or 


z(ti)  «  *1  «  A?.1/2. 

\J2U 


v(t.) , . *&).. 

i-p(ti) 

d>  q 


Grouped  Data 

1.  Data  for  input  are: 

(a)  A  set  of  ordered  values  x^,  the  number  of  trials  at  level  xj, 
and  the  number  of  successes  that  level. 


where  ^  and  £T nay  have  affixes  denoting  them 
as  provisional  or  maximum  likelihood  solutions. 


Ordinate  of  the  normal  density 


(See  Inclosure  1.) 


A  constant  chosen  for  each  set  of  data  (In 
the  uuite  of  x)  to  halt  the  Iteration  process. 

lb;  process  halts  when  ^  lAff’l  <. 
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xi  -  level  of  stimulus 

■  number  of  trials  at  level  xi 
-  number  of  successes  at  level  x^> 
xi  nj  «i 

••• 

etc. 

(b)  A  first  guess  (p.0yr.  )  of  the  unknown  parameters.  These  can 
usually  be  estimated  from  a  plot  of  the  data  on  normal 
probability  paper. 

(c)  A  value  d. 

NOTE:  The  program  will  take  a  maximum  of  15  values  of  x.  These  need 
not  be  equally  spaced. 

A  A 

2.  /-L  and  C are  calculated  by  the  following  process: 


(a)  For  each  calculate 


-  (*i  -  M„)/o ; 

v(ti)  and  v(-ti) 

Form  the  following  sums 

a  -  X  (iH-m!)  v(ti) 

a'  ■ 

X  v(-ti) 

b  -  X  (ni-mi)  ti  v(4) 

b'  - 

X  ti  v(-ti) 

c  -  £(ni-wi)  [w(ti)]2 

c '  ■ 

X® i  [w(-ti)]2 

d  ■  X (ni“mi)  t*2  v(ti) 

d*  . 

£mi  t^  v(-ti) 

e  .  X(ni*“i)  ti  [wCti)]2 

e'  ■ 

i  ®i  ti  Lv(-ti)]2 

f.  X(ni-n1)  ti3  v(t[) 

f  . 

£  Ki  ti3  w(-ti) 

E  -  ^(ni-®!)  ti£  Lv(tl)]2 

C*  - 

I  ®i  ti2  ,  v(-ti)j: 
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(c)  Solve  the  following  2  equations  for  4^- and  hG  . 


(a-a')/r 


b  -  c  •  b*  -  c' 


A  d-e-a-d'-e' 

& +  - —-r -  a<r 


-(b-b')/^  .  d,~  *  -  »  -,.r  a0- 

0”#  0, 

(d)  Then  y.  «  a,m- 

O’,  ■  ♦  A|C* 

(e)  Repeat  steps  (a),  (b)  and  (c)  with  replaced  by  . 

Continue  the  Iteration  until  the  stopping  rule  halts  the  iteration.  The  last 
values  ^ « .jj,  arc  taken  as  p. ; 

4  * 

3*  Precision  of/~  <T  is  calculated  as  follows: 

(a)  For  each  xi  calculate 

ti  »  (*i  -  P-  )/fr 
zi#  w(ti)  and  w(-ti) 

(b)  Fora  the  following 

Ini  zi(v(ti)  +  w(-tij] 


Jnj  tt  SiCvftj)  ♦  wf-tij] 

o  ■  I 

c  -  i°i  *!  h[v(h)  ♦  w^j) 


AC  -  32  .  D 

vw  £  •  c/d 

vmr  ■  a/D 


A A  , 

•9f£f  •  -B/D 


62-AL-15fc 

7 


4.  The  following  results  are  typed  out  by  the  computer. 

(a)  p-o.C 1  and  successive  provisional  solutions,  as  veil  as  successive 

values  4k  p.  and  . 

(b)  D,  wjt  ,  and  eor^f  ■  . 

Ungrouped  Data,  2MB 

1.  Data  for  Input  are: 

(a)  A  set  of  unordered  values  xi  and  an  associated  indicator  of  "success" 
or  "failure"  dj.*  4”  ■  1  if  success,  4  *  0  If  failure. 

»1  <fi 


etc. 

(b)  A  first  guess  (/-*.,<£;  )  of  the  unknown  parameters  (optional).  If 
no  special  first  guess  is  desired,  the  computer  makes  a  first 
estimate  from  the  data. 

(c)  A  value  d. 

2.  p-  and  are  calculated  by  the  following  procccs: 

(a)  Examine  xt  to  find  xy  and  x^* 

M-o-  (*n  +  xl)/2 

“  *1* 

(b)  for  each  x^  calculate 

ti  •  -  ^)/<r0 

v(tj)  for  x^  having  0 

v(.t1)  far  X*  having  4-1. 
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Form  the  folloving  sums 

For  x's  haring  4*0 

For  x's  haring  &  ■  1 

a  -  I  v(ti) 

a'  -  X  ¥(-4) 

*  -  T  *i  w(*i) 

b‘  -  ZH  v(-ti) 

c  «  E^v(ti)]2 

c'  -  f  [v(-ti)]2 

d  -  Z.ti2  w(tt) 

d*  «  I  ti2  v(-ti) 

e.  JtjLvftj)]2 

e'  •  £.ti  L  vf-tj)]2 

f  -  Z  ti3  w(tt) 

f  -  £  ti3  ^(-ti) 

6-1  ^12[ 

s'  -  2  ti2  l  f 

(d)  Soxre  the  following  2  equations  for  Ajm.  and  AC  . 

(«  -  a')/,  .  i-;  c  -  b’  -  c'  -  .  *  d  ...  a  -  a- 


0>  -  b-)/fi 


A|a. 

^ —  V  1 

<7. 

e  -  a  -  d1  -  e*  ♦  a* 

<rv 


■A;*- 


♦  f  .  a  .  e  .  r  ♦  a1  .  g1 


*<r 


(e)  Then  ■  p~N  + 

<r,  -  <r,  ♦  <r 

(f)  Repeat  otepa  (b),  (c)  and  (d)  v\th  replaced  by  f*t ,<T,  . 

Continue  the  iteration  until  the  stopple  rule  halts  the  iteration.  The  laet 
values  Cjj  are  taken  as  £  ^  . 

3.  Precision  of  m- ^  is  calculated  as  follows :  (optional) 

(a)  For  each  xi  calculate 

*1  -  (*1  •  A  )/£* 

*i»  "(ti)  and  ^(-ti) 


62-AL-154 

9 


(b)  Fora  the  following 

A  -  I»j  [v(t^)  4  w(-fcj)] 

B  .  ♦  v(-ti)] 

F1 

c  m  jtf  »1  [v(tj)  ♦  v(-t1)] 

D  -  AC  -B2 

ver  p  -  c/D 
var  d-  •  A/D 
-  -B/D 

V.  The  following  results  are  typed  out  by  the  computer 

(a)  * 6{T%  and  successive  provisional  solution,  as  well  as  (successive 

values  £  ^  and  AC  . 

(b)  D,  var  w  T  undeov^r.  (optional) 

Unsrouped  Data.  Bo  OH  (Expressed  as  a  nodiflcatlon  of  the  preceding  proproc) 

1.  Data  for  Input  are: 

(a)  So  change. 

(b)  An  assused  or  known  value  of  <T .  %  is  eotlaatod  by  the  computer. 

(c)  So  change. 

2.  A  is  calculated  by  the  following  process: 

(a)  Quit  est  irate  of  C\. 

(b)  Replace  C ,  by  C“. 

(c)  Calculate  only  a,  a*,  b.  b',  c  and 
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(d)  Jolve  for  Af>  . 

.  (a  -  a')  «  ±S.l^y:^LSl. 


A  f>- 


(s)  and  (f)  Otolt  reference  to  (j . 

3.  Precision  of  p-  Is  calculated  as  follows:  (optional) 

(a)  No  change  except  that  (T  •  C  . 

(b)  Calculate  only  the  fuantltles  A  and  l/l. 

k.  The  following  results  are  typed  out  by  the  computer. 

(a)  and  successive  provisional  solutions  as  well  as  values  of 

(b)  ■  l/A.  (optional) 


The  estlaation  procedure  used  in  this  prograa  for  statistical  estimation 
of  unknown  paroaeters  has  a  msebsr  of  desirable  properties  (unbiased,  plnlsam 
varlonee  estlaatee)  when  the  nusber  of  trials  Is  large.  Relatively  little  is 
known  froa  a  purely  aathcautlcal  point  of  view  about  the  estimates  when  the 
□writer  of  trials  Is  aw  11  ■  Results  of  a  nuruber  of  investigations  using 
slaulated  trials  as  well  as  experimental  data  indicate  that  the  estlaatee  of  f*. 
are  unbiased  for  practical  purposes  but  that  estlaatee  of  f on  the  average 
are  too  anil.  (See  first  Report  on  Ord  Froject  TBb-005B,  Rpt  So.  &FB»?B3»i0/i 
Aid  July  1  •'it  for  e  discussion  of  soee  of  the  results  of  various  cceqparlsons.) 

It  is  believed  that  estlaatee  of  p.  con  be  used  unhesitatingly  even  when  staple s 
are  very  sail.  Estlactcs  of  (T ,  however,  are  not  only  biased  but  ere  sxtrealy 
variable  under  such  conditions  and  aust  be  used  with  utnoet  caution,  ftlnoe  the 
estlaatee  of  precision  of  h-  and  C  are  proportional  to  9" ,  the  statistioal 
variability  of  the  latter  le  reflected  In  them.  Guidelines  for  the  use  of  these 
stetlstlca  for  very  sail  staples  have  not  a  yet  been  developed,  (for  as  11 
staples  the  experimental  technique,  euch  as  u^ and* down,  is  probably  mare 
Influential  in  deteminlw  tJ.ce*  statistics  than  Is  the  estlaation  procedure.) 

An  essential,  preliminary  step  in  the  application  of  the  prone  dure  to 
saall  aaples  is  a  careful  examination  of  the  data.  First,  if  00  sons  of  alsad 
results  exists,  no  estimate  of  T  (other  than  scro)  cos  be  obtained.  Second,  the 
probability  of  succsa  le  an  increasing  function  cf  the  stlaulus  x.  therefore, 
if  the  highest  vula  of  x  resulted  In  e  failure,  or  the  lowest  value  of  x  resulted 
in  e  success,  att tapis  to  estlate  C  will  generally  oe  futile.  The  process 
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usually  diverges  with  O’  increasing  vlthout  Halt.  This  situation  is  the  result 
of  incomplete  testing  (sometimes  unavoidable)  and  can  not  be  rectified  at  the 
data  analysis  stage.  Third,  if  there  is  no  3G  and  a  value  of  cr  is  a caused  in 
order  to  obtain  an  estimate  of  /^,  the  estimate  of  the  precision  of  pk  should 
not  be  relied  upon  since  it  is  proportional  to  the  asstned^f.  In  other  vards, 
choice  of  a  different  value  for  cr  vould  probably  change  pi  very  little  but 
would  change  its  precision  estimate . 

Although  the  computer  program  was  developed  for  the  analysis  of  test 
data,  it  also  provides  a  means  of  conducting  further  vark?  employing  simulated 
experiment a  from  known  distributions  to  determine  properties  of  this  estimation 
procedure  for  small  samples. 


EXAMPLE 

Two  examples  are  given  to  illustrate  the  use  of  this  program.  The  first 
is  taken  from  a  test  to  determine  fuze  arming  distance  and  uses  the  program 
for  grouped  data.  The  second  is  from  a  test  of  a  projectile  for  penetration  of 
armor  and  is  an  example  of  ungrouped  data  having  a  zone  of  mixed  results. 

Example  1.  The  following  data  was  obtained  in  the  experiment. 


Distance  from 

Kux&le  to  Target,  in. 

No.  of  Rd 
Fired 

No.  of  Fuzes 
Armed 

575 

9 

1 

581 

10 

3 

587 

10 

3 

593 

10 

5 

602 

10 

6 

606 

8 

6 

These  data  are  fed  to  the  computer  vith  the  distances  os  xj,  the  number  of 
rounds  fired  as  nj,  and  the  number  of  fuz.ee  jartned  as  Z14.  k  plot  of  ratios, 
mi/ni  on  normal  probability  paper  suggests  ,»<.*  »  596  and  C  *  18.  1  was  chosen 

as  0.1.  The  computer  types  the  following  results.  (The  computer  automatically 
prints  7  decimals  in  all  results  unless  special  instructions  are  inserted.) 


1 

^  i 

0 

596.0000000 

.1.2096147 

18.0000000 

.6334687 

1 

59^.7903852 

-  .0100556 

18.6334687 

.3960649 

2 

59^7803295 

,0016863 

19.0295337 

.0265457 

p.  - 

var  p-  • 
var  $■  ■ 

COY  9 

594.7820159 

12.6976265 

39.18^7015 

7.2206360 

cr  •  19.0560795 

The  standard  errors  of  /I  and  T  ore  3.5 6  and  6.26  respectively. 
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Often  a  confidence  Interval  for  a  given  percentile  is  desired.  Suppose 
the  5th  percentile  is  to  be  estimated  as  well  as  a  confidence  interval  for  ii„ 

A  ^ 

«*1.64  CT  •  563.5 

a  A 

Since  {*■  and  <T  are  jointly  normally  distributed  with  variances  and  covariance 
as  calculated  j  bcre,  is  also  normally  distributed. 

Var  P5  «  rsr  p  +  var  <3*  (1.64)2  -  2  (1.64)  cov/^J". 

<+  .  , 

Substitution  of  calculated  valuer.  gives  vnr  P5  b  and  standard  deviation 

of  P5  a  y.71.  Thus  the  interval  can  be  written  as 

563.5  -  19.71  <  P5  <  563.5  +  *9.71 

where  K  is  chosen  from  normal  tables  to  give  the  desired  confidence  coefficient. 
For  example,  K  •  1.96  for  a  coefficient  of  95$,  and 

544.5  C  p5  c  582.5. 

Example  2.  The  following  data  were  obtained  in  a  penetration  test  of  a 
projectile. 


Striking  Velocity 
of  Projectile,  fps 

636 

769 

628 

644 

812 

Or.% 

893 

859 


Results 

partial  penetration 


«:  n 

complete  " 

(I  ft 

partial  " 

ft  i» 


complete 


The  data  are  transcribed  for  the  computer  as 


Si 


636  0 

769  0 

826  1 

644  1 

812  0 

859  0 

893  1 

859  1 


ht>P'osu'e  estimated  by  the  computer „  d  ic  taken  as  0.1. 


Th«  computer  types  the  following  table 
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1 

-  Ki  A.  ^ 

Ol 

0 

1 

841.5000000  -IO.8213909 

830.6786090  -  .6187W3 

35.0000000 

34.0415644 

2 

830.0598606  -  .1775926 

37.7590319 

3 

829.8822680  -  .0131477 

38.9080322 

4 

829.8691202  -  .0000580 

38.9961012 

p>  m  829.8690622  c » 

38.9965391 

Tar 

£  -  481.5176056 

var 

-  800.6888854 

COT 

Jt.fr  -  -147.0445795 

A;4,0r 


.9581*355 

3.7175175 

1.11*69502 

.0880689 

.0004379 


1  lad 

v(t)  functions 
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ximation  of  Certain  Functions  in  BRL  Technical  Note  151 


In  maximum  likelihood  procedures  for  estimation  of  parameters  mi  and  si'^aa 
of  the  normal  curve  from  sensitivity  data  functions  z/P  and  z/Q  are  needed. 

Since  P  and  Q  are  definite  integrals,  their  repeated  calculation  is  time 
consuming.  Consequently,  for  machine  programing  of  the  procedure  approximations 
that  reduce  the  calculations  have  been  developed. 


r(t)  .  1  e-‘2/2 

■{zi 


p(t)  -  J  z (x)  dx 

-  oO 

v(t)  .  z(t)  v(t)  -  Z/Q,  and  w(-t)  -  Z/P  in  BRL  Note  151 

1  -  P(t) 


w(t)  •  .3989^2  e“t2/2 
v(t)  -  c0  +  c^t  +  c2t2  ♦  c3t3 


co 

C1 

c2 

c3 

2.5 

•1.5 

.89LLI+63 

.6428656 

.2727600 

.03014958 

1.5 

-1.0 

.8187139 

.7014965 

.1801986 

.00901483 

1.0 

*  *3 

.7995 070 

.6409503 

.U89404 

-.01111111 

.3 

1.0 

.7979051+ 

.6365250 

.1084799 

-.01780600 

1.0 

2.5 

.781+789!+ 

.6702693 

.0780463 

-.008037033 

2.5 

5.0 

.6873630 

.7831186 

.03346049 

-.002025425 

5.0 

00 

.18629 

1.000000 

0 

0 

Incl  1 


